Bidirectional Online Probabilistic Planning

We present Bidirectional Online Probabilistic Planner (BOPP)1, a novel planner that combines elements of Decision Theoretic Planning(DTP) and forward search. In particular, BOPP uses a combination of SPUDD and Upper Confidence Trees(UCT). We present our approach and some experimental results on the domains presented in the boolean fluents MDP track of the International Probabilistic Planning Competition(IPPC) 2011. Decision Theoretic Planning (DTP) (Boutilier, Dean, and Hanks 1999) is a well established method for solving probabilistic planning domains by casting them as Markov Decision Processes (MDP) and generating a policy. Classical solutions to DTP (Bellman 1957; Howard 1960) require the entire state space to be enumerated. This approach is usually infeasible for solving planning problems of interest. Recent advances in DTP have mitigated this effect by providing solution algorithms for factored (Boutilier, Dearden, and Goldszmidt 1999) and relational (Boutilier, Reiter, and Price 2001) MDPs. These solutions are abstract and require enumeration only of the relevant conditions that create a partition of the state space into equivalence classes (based on the policy or value), instead of enumerating the entire state space. One factored MDP solver in particular, SPUDD (Hoey et al. 1999), has been very successful at solving planning problems and has spawned numerous variants over the last decade. SPUDD employs Algebraic Decision Diagrams to represent and solve the underlying MDP using value iteration. However, experiments have shown that SPUDD proves to be inefficient for many of the planning problems presented in the recent IPPC. Forward Search has been another classic approach for AI planning. Brute force search is typically infeasible for large state spaces because the size of the search tree is exponential in the depth (length of the plan). Planners based on heuristic search, however, have shown success at the recent planning competitions (Bonet and Geffner 2001; Yoon, Fern, and Givan 2007; Teichteil-Koenigsbuch, Infantes, and Kuter 2008). The heuristic values of states or state-action pairs are typically derived automatically by solving a relaxation to the probabilistic planning problem. More recently, search algorithms for probabilistic planning based on simulation and